The present invention relates to data storage systems, and more particularly, this invention relates to hierarchical or “tiered” storage-based systems capable of being used in high performance, redundant data systems.
Television has become a mainstay of society around the world. The ability to record television programming has proliferated in recent years, and particularly in conjunction with digital video recorders (DVRs) such as TIVO brand DVRs. Typically, a subscriber has one or more DVRs present in his or her home, that allow recording of television programming as it is received from the broadcaster, and playback on demand.
In an effort to improve efficiency and reduce cost, a remote storage digital video recorder (RS-DVR) may be used to store huge amounts of video data on a network site, which would essentially provide the same functionality as a local digital video recorder. Like a local DVR, a user of a RS-DVR programs into the RS-DVR system which programs are to be recorded, and plays the recorded programs back when desired. During playback, the user can use any of the trick modes (e.g., pause, fast forward, fast reverse, etc.), and content providers require support for these trick modes. For example, disk-based streaming systems which store and stream programs from a hard disk drive may require additional processing and storage on the part of the server when trick modes are used, because separate files for fast forward and rewind may need to be stored. The user decides when a recording is to be deleted. The only difference from a user's point of view is that the RS-DVR's storage is physically remote, e.g., it is at the content provider's end of the connection, not at the user's end of the connection, as is the case with conventional local DVRs.
A RS-DVR is a lower cost storage solution for a content provider to maintain versus each user having a local DVR, because it costs less to deploy, administer, and maintain a centralized storage resource, as opposed to a content provider deploying distributed storage at each user's access point (e.g. residence, workplace, mobile hotspots, etc.). It also costs less for specialists to service a centralized Information Technology (I/T) facility than to service multiple local DVRs deployed at user's access points (which can be physically altered and/or damaged by the user).
One issue plaguing the implementation of RS-DVR services is the need to provide fast access to huge amounts of data to multiple users at once. Moreover, regulations in some jurisdictions may require each subscriber to have ownership of his or her own copy of a recorded program, where ownership is some association between the subscriber, device of the subscriber, etc., and a given copy of the recorded program. As apparent, the required data capacity could be astronomical. Implementation of higher speed storage systems, such as hard disk drives, in an RS-DVR work well, but the high cost of an all-disk system makes such systems unaffordable. What is therefore needed is a way to provide a combination of high performance coupled with low storage cost per unit of data.
One approach previously deemed too slow for high performance, high demand systems such as RS-DVRs is storage hierarchical storage management (HSM) systems. Hierarchical storage, with active files on a first tier of storage media (such as hard disk, rewritable optical disk, nonvolatile memory, etc.) and archived files on a second storage tier of less expensive and/or slower-to-access storage media (such as magnetic tape, digital tape, hard disk, optical disk, etc.) is popular for slower data applications for its cost savings, energy savings, etc. A common scheme throughout HSM systems is to use hard disk media for a first storage tier and magnetic tape media for a second storage tier, however any type of media may be used. In some HSM systems, random access storage media, such as hard disk media, is predominantly used in the first tier, while sequential access storage media, such as magnetic tape media, is predominantly used in the second tier. However, traditional HSM systems suffer from several drawbacks which limit their adoption, particularly in high performance systems such as RS-DVRs.
One problem with using standard HSM for a RS-DVR application is that data may need to be moved from the lower, slower tier (e.g. tape) to the higher, faster tier (e.g. disk) very quickly. Standard HSM operation results in too much latency for these high performance environments. For example, when it comes time to access data which has been moved to tape, it can take about 10 seconds to mount the tape cartridge, 15 seconds to load-thread the tape, and 95 seconds or longer to locate the start of the data, which might be located at the farthest end of the tape. In some instances, a worst case read access time of up to about 2 minutes can be encountered, which is unacceptable in high performance environments such as video playback. Since users typically expect that when a program is chosen and “play” is selected, that the program will begin to play expediently, any significant delay to accessing the program is unacceptable to the service provider. The result is that standard HSM systems have heretofore been thought too slow for use in RS-DVR applications.